Papers with writing tasks
How Reliable Are AI-Generated-Text Detectors? An Assessment Framework Using Evasive Soft Prompts (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to detect AI-generated text are inadequate, causing misuse of the text. |
| Approach: | They propose a universal evasive prompt framework that can prompt any PLM to generate “human-like” text that can mislead detectors. |
| Outcome: | The proposed approach can prompt any PLM to generate “human-like” text that can mislead detectors. |
CollabStory: Multi-LLM Collaborative Story Generation and Authorship Analysis (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies on LLM-LLM collaboration for open-ended tasks have focused on human-LLm interaction. |
| Approach: | They propose to generate a dataset exclusively for LLMs to explore multi-LLM collaboration scenarios . they extend their authorship-related tasks for multi-llm settings and extend their baselines . |
| Outcome: | The authors extend authorship-related tasks for multi-LLM settings and present baselines for LLM-LLMS collaboration. |
People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text (2025.acl-long)
Copied to clipboard
| Challenge: | Qualitative analysis of experts’ free-form explanations shows that while they rely heavily on specific lexical clues (‘AI vocabulary’), they also pick up on more complex phenomena within the text (e.g., formality, originality, clarity). |
| Approach: | They hire annotators to read 300 non-fiction English articles, label them as either human-written or AI-generated, and provide paragraph-length explanations for their decisions. |
| Outcome: | The annotators who frequently use LLMs for writing tasks outperform commercial and open-source detectors even without evasion tactics like paraphrasing and humanization. |
Small Language Models can Outperform Humans in Short Creative Writing: A Study Comparing SLMs with Humans and LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | a fine-tuned small language model (SLM) can generate human-like text, but it requires immense computational resources and large datasets. |
| Approach: | They evaluate the creative writing abilities of a fine-tuned small language model, BART-large . they compare it to human writers and two large language models: GPT-3.5 and GPT-4o . |
| Outcome: | The proposed model outperforms human writers and two large language models in two experiments . the results highlight how model size and fine-tuning influence creativity, fluency, and coherence . |
LLMCrit: Teaching Large Language Models to Use Criteria (2024.findings-acl)
Copied to clipboard
| Challenge: | Current research on using criteria to provide feedback on tasks is limited . a general framework that can be used to teach large language models to use criteria is lacking . |
| Approach: | They propose a framework that enables large language models to use criteria for feedback . criteria are extracted from guidelines and construct in-context demonstrations for each criterion . |
| Outcome: | The proposed framework can be used to provide natural language feedback on tasks. |
NewsBench: A Systematic Evaluation Framework for Assessing Editorial Capabilities of Large Language Models in Chinese Journalism (2024.acl-long)
Copied to clipboard
Miao Li, Ming-Bin Chen, Bo Tang, ShengbinHou ShengbinHou, Pengyu Wang, Haiying Deng, Zhiyu Li, Feiyu Xiong, Keming Mao, Cheng Peng, Yi Luo
| Challenge: | a novel evaluation framework assesses the capabilities of Large Language Models (LLMs) for editorial capabilities in Chinese journalism. |
| Approach: | They propose to use a benchmark dataset to assess the capabilities of Large Language Models (LLMs) for editorial capabilities in Chinese journalism. |
| Outcome: | The proposed evaluation framework is based on a dataset of 1,267 test samples in 24 news domains. |
Rationalize and Align: Enhancing Writing Assistance with Rationale via Self-Training for Improved Alignment (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing writing assistants rely on supervised fine-tuning to optimize models for multiple revisions. |
| Approach: | They propose a framework that enhances WA performance with rationale and alignment. |
| Outcome: | The proposed framework outperforms state-of-the-art WAs and the closed-source GPT-4o by 3.9 and 7.1 points on average across eight well-established writing-related test sets. |
Outcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Models (2026.acl-long)
Copied to clipboard
Binghai Wang, Yantao Liu, Yuxuan Liu, Tianyi Tang, Shenzhi Wang, Chang Gao, Chujie Zheng, Yichang Zhang, Le Yu, Shixuan Liu, Tao Gui, Qi Zhang, Xuanjing Huang, Bowen Yu, Fei Huang, Junyang Lin
| Challenge: | Recent studies observe a phenomenon where reward models achieve high accuracy on static datasets but fail to generalize effectively during RLHF. |
| Approach: | They propose a method that combines rationale consistency with outcome accuracy to improve performance on RM-Bench and JudgeBench. |
| Outcome: | The proposed method surpasses baselines on RM-Bench and JudgeBench by an average of 5% and improves creative writing tasks by 7%. |
Can You Make It Sound Like You? Post-Editing LLM-Generated Text for Personal Style (2026.acl-long)
Copied to clipboard
| Challenge: | Despite the growing use of large language models for writing tasks, it remains unclear whether users can effectively reshape LLM-generated text to reflect their personal style. |
| Approach: | They conduct an online study in which participants post-edit LLM-generated drafts for writing tasks where personal style matters to them. |
| Outcome: | The results show that post-editing increases stylistic similarity to unassisted writing and reduces similarity with fully LLM-generated output. |